We develop novel statistical techniques for nonparametric Bayes analysis of high-dimensional covariate data, directly motivated by the largest population-based study ever conducted on the causes of birth defects. The methods we develop will enable borrowing of information and shrinkage across high-dimensional environmental, biomedical, pharmacological, and sociodemographic risk factors (and interactions among them) and across a multitude of birth defects, many of which are too rare to be studied in isolation. Using a hierarchical structure directly motivated by embryonic development, the borrowing of information can be informed by our knowledge of mechanistic development of the embryo. These novel methods may significantly impact the study of rare congenital malformations. The methods to be developed have broad application in public health and medicine, where exposures or characteristics of interest may be great in number and interactions are important, such as the examination high-dimensional gene by environment and gene-gene interactions.